Liveness test method and liveness test apparatus

ABSTRACT

Disclosed is a liveness test method and liveness test apparatus. The liveness test method includes determining a presence of a subject using a radar sensor, performing a first liveness test on the subject based on radar data obtained by the radar sensor, in response to the subject being present, acquiring image data of the subject using an image sensor, in response to a result of the first liveness test satisfying a first condition, and performing a second liveness test on the subject based on the image data.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit under 35 USC § 119(a) of KoreanPatent Application No. 10-2019-0165777 filed on Dec. 12, 2019 in theKorean Intellectual Property Office, the entire disclosure of which isincorporated herein by reference for all purposes.

BACKGROUND Field

The following description relates to technology for testing a livenessof an object.

2. Description of Related Art

In a user authentication system, a computing device determines whetherto allow an access to the computing device based on authenticationinformation provided by a user. In an example, the authenticationinformation includes a password input by the user or biometricinformation of the user. The biometric information includes informationrelated to features such as, a fingerprint, an iris, or a face.

Recently, there is a growing interest in face anti-spoofing technologyas a security method for user authentication systems. Face anti-spoofingverifies whether a face of a user input into the computing device is afake face or a genuine face. For this, features such as Local BinaryPatterns (LBP), Histogram of Oriented Gradients (HOG), and Difference ofGaussians (DoG) are extracted from the input image, and whether theinput face is a fake face is determined based on the extracted features.Face spoofing is in the form of attacks using a photo, a video, or amask. In face authentication, it is important to identify such attacks.

SUMMARY

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter.

In one general aspect, there is provided a liveness test method,including determining a presence of a subject using a radar sensor,performing a first liveness test on the subject based on radar dataobtained by the radar sensor, in response to the subject being present,acquiring image data of the subject using an image sensor, in responseto a result of the first liveness test satisfying a first condition, andperforming a second liveness test on the subject based on the imagedata.

The liveness test method may include determining whether there may be ahuman face based on the radar data.

The radar data may be obtained using a portion of antennas in the radarsensor.

The determining may include extracting an intensity feature of aby-distance reception signal from the radar data, and determining thepresence of the subject based on the intensity feature.

The determining may include continuously obtaining data from the radarsensor, and determining whether there may be a subject based on thecontinuously obtained data.

The performing of the first liveness test may include extracting afeature from the radar data, and determining a result of the firstliveness test on the subject based on the extracted feature.

The extracting may include extracting, from the radar data, any one orany combination of a distance to the subject, a size of the subject, adirection in which the subject may be positioned, and a shape of thesubject.

The acquiring may include activating the image sensor, in response tothe result of the first liveness test satisfying the first condition,and acquiring the image data from the activated image sensor.

The performing of the second liveness test may include detecting a faceregion of the subject in the image data, and performing the secondliveness test based on the face region.

The detecting may include detecting the face region in the image databased on the radar data.

The liveness test method may include performing a third liveness test onthe subject based on the radar data and the image data, in response to aresult of the second liveness test satisfying a second condition.

The performing of the third liveness test may include extracting a firstfeature based on pixel values of pixels included in a face region in theimage data, obtaining another radar data using the radar sensor,extracting a second feature from the another radar data, and determininga result of the third liveness test based on the first feature and thesecond feature.

The another radar data may be obtained using a plurality of polarizationantennas of the radar sensor.

The another radar data may be obtained for each of a plurality ofchannels using the radar sensor, and the extracting of the secondfeature may include extracting a channel-based signal feature from theanother radar data.

In another general aspect, there is provided a liveness test method,including determining a presence of a subject using a radar sensor,acquiring image data of the subject using an image sensor, in responseto the subject being present, and performing a first liveness test onthe subject based on the image data.

The determining may include continuously obtaining radar data from theradar sensor, and determining whether the subject may be present basedon the obtained radar data.

The acquiring may include activating the image sensor, in response tothe determination that the subject may be present, and acquiring theimage data from the activated image sensor.

The liveness test method may include performing a second liveness teston the subject based on radar data obtained by the radar sensor and theimage data acquired by the image sensor, in response to a result of thefirst liveness test satisfying a first condition.

The performing of the second liveness test may include extracting afirst feature based on pixel values of pixels included in a face regionin the image data, obtaining another radar data using the radar sensor,extracting a second feature from the another radar data, and determininga result of the second liveness test based on the first feature and thesecond feature.

In another general aspect, there is provided a liveness test apparatus,including a radar sensor, an image sensor, and a processor configured todetermine a presence of a subject using the radar sensor, perform afirst liveness test on the subject based on radar data obtained by theradar sensor, in response to the subject being present, acquire imagedata of the subject using the image sensor, in response to a result ofthe first liveness test satisfying a first condition, and perform asecond liveness test on the subject based on the image data.

The processor may be configured to continuously obtain data from theradar sensor, and to determine the presence of the subject based on theobtained data.

The processor may be configured to activate the image sensor, inresponse to the result of the first liveness test satisfying the firstcondition, and to acquire the image data from the activated imagesensor.

The processor may be configured to perform a third liveness test on thesubject based on the radar data and the image data, in response to aresult of the second liveness test satisfying a second condition.

The radar sensor may be configured to operate while being included in acommunication module.

In another general aspect, there is provided a liveness test apparatus,including a radar sensor, an image sensor, and a processor configured todetermine whether a subject may be present using the radar sensor,acquire image data of the subject using the image sensor, in response tothe subject being present, and perform a first liveness test on thesubject based on the image data.

The processor may be configured to perform a second liveness test on thesubject based on radar data obtained by the radar sensor and the imagedata acquired by the image sensor, in response to a result of the firstliveness test satisfying a first condition.

In another general aspect, there is provided a liveness test method,including determining a presence of a subject using a radar sensor,performing a first liveness test on the subject based on first radardata obtained by the radar sensor, in response to the subject beingpresent, acquiring image data of the subject using an image sensor, inresponse the first liveness test satisfying a first threshold,performing a second liveness test on the subject based on the imagedata, performing a third liveness test on the subject based on secondradar data and the image data, in response the second liveness testsatisfying a second threshold.

A number of antennas of the radar sensor used to obtain the second radardata may be greater than a number of antennas of the radar sensor usedto obtain the first radar data.

A number of antennas of the radar sensor used to obtain the first radardata may be greater than a number of antennas of the radar sensor usedto determine the presence of the subject.

Other features and aspects will be apparent from the following detaileddescription, the drawings, and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1 and 2 illustrate examples of a biometric authentication and aliveness test.

FIG. 3 illustrates an example of an electronic device with a radarsensor and a camera sensor.

FIG. 4 illustrates an example of a liveness test process.

FIG. 5 illustrates an example of controlling the activation of an imagesensor based on radar data.

FIG. 6 illustrates an example of detecting a face region in image data.

FIGS. 7A and 7B illustrate examples of performing a third liveness test.

FIG. 8 illustrates an example of a liveness test method.

FIG. 9 illustrates an example of a liveness test method.

FIG. 10 illustrates an example of a process of training a liveness testmodel.

FIG. 11 illustrates an example of a configuration of a liveness testapparatus.

FIG. 12 illustrates an example of a configuration of an electronicdevice.

Throughout the drawings and the detailed description, unless otherwisedescribed or provided, the same drawing reference numerals will beunderstood to refer to the same elements, features, and structures. Thedrawings may not be to scale, and the relative size, proportions, anddepiction of elements in the drawings may be exaggerated for clarity,illustration, and convenience.

DETAILED DESCRIPTION

The following detailed description is provided to assist the reader ingaining a comprehensive understanding of the methods, apparatuses,and/or systems described herein. However, various changes,modifications, and equivalents of the methods, apparatuses, and/orsystems described herein will be apparent after an understanding of thedisclosure of this application. For example, the sequences of operationsdescribed herein are merely examples, and are not limited to those setforth herein, but may be changed as will be apparent after anunderstanding of the disclosure of this application, with the exceptionof operations necessarily occurring in a certain order. Also,descriptions of features that are known may be omitted for increasedclarity and conciseness.

The features described herein may be embodied in different forms, andare not to be construed as being limited to the examples describedherein. Rather, the examples described herein have been provided merelyto illustrate some of the many possible ways of implementing themethods, apparatuses, and/or systems described herein that will beapparent after an understanding of the disclosure of this application.

The terminology used herein is for the purpose of describing particularexamples only, and is not to be used to limit the disclosure. As usedherein, the singular forms “a,” “an,” and “the” are intended to includethe plural forms as well, unless the context clearly indicatesotherwise. As used herein, the term “and/or” includes any one and anycombination of any two or more of the associated listed items. As usedherein, the terms “include,” “comprise,” and “have” specify the presenceof stated features, numbers, operations, elements, components, and/orcombinations thereof, but do not preclude the presence or addition ofone or more other features, numbers, operations, elements, components,and/or combinations thereof.

In addition, terms such as first, second, A, B, (a), (b), and the likemay be used herein to describe components. Each of these terminologiesis not used to define an essence, order, or sequence of a correspondingcomponent but used merely to distinguish the corresponding componentfrom other component(s). Although terms of “first” or “second” may beused to explain various components, the components are not limited tothe terms. These terms should be used only to distinguish one componentfrom another component. For example, a “first” component may be referredto as a “second” component, or similarly, and the “second” component maybe referred to as the “first” component within the scope of the rightaccording to the concept of the present disclosure.

Throughout the specification, when an element, such as a layer, region,or substrate, is described as being “on,” “connected to,” or “coupledto” another element, it may be directly “on,” “connected to,” or“coupled to” the other element, or there may be one or more otherelements intervening therebetween. In contrast, when an element isdescribed as being “directly on,” “directly connected to,” or “directlycoupled to” another element, there can be no other elements interveningtherebetween. Likewise, expressions, for example, “between” and“immediately between” and “adjacent to” and “immediately adjacent to”may also be construed as described in the foregoing.

Hereinafter, examples will be described in detail with reference to theaccompanying drawings. Like reference numerals in the drawings denotelike elements, and thus their description will be omitted.

FIGS. 1 and 2 illustrate examples of a biometric authentication and aliveness test.

A biometric authentication is authentication technology using personalbiometrics such as a fingerprint, an iris, a face, veins, skin, amongauthentication technologies for user verification. In the biometricauthentication, a face verification determines whether a user is a validuser based on face information of the user attempting an authentication.The face verification is used to authenticate a valid user for a userlog-in, a payment service, and an access control.

Referring to FIG. 1 , an electronic device 120 performs anauthentication process with respect to a user 110 attempting to accessthe electronic device 120 through a biometric authentication. Theelectronic device 120 senses an approach of the user 110 using a radarsensor 130 included in the electronic device 120 and, in response to thedetermination that the user 110 approaches within a distance, performs abiometric authentication (for example, face verification) process withrespect to the user 110. The electronic device 120 automaticallyperforms the biometric authentication process using the radar sensor130, even if the user 110 does not perform a separate manipulation tostart the biometric authentication process such as, for example,pressing a button or touching a screen.

In an example, the electronic device 120 performs the biometricauthentication process based on radar data obtained using the radarsensor 130 and/or image data acquired using an image sensor 140 such asa camera. The electronic device 120 determines an authentication resultby analyzing the radar data and/or the image data. The biometricauthentication process includes, for example, a process of extractingfeatures from the radar data and/or the image data, comparing theextracted features to enrolled features related to a valid user, anddetermining whether an authentication is successful based on thecomparison. For example, if the electronic device 120 is being locked,the electronic device 120 may be unlocked in response to thedetermination that an authentication with respect to the user 110 issuccessful. In another example, when it is determined that theauthentication of the user 110 has failed, the electronic device 120 maycontinue to remain locked.

A valid user enrolls his/her biometric features in the electronic device120 in advance through an enrollment process, and the electronic device120 stores information to be used to identify the valid user in astorage device or cloud storage. For example, a face image of the validuser or face features extracted from the face image are stored asenrolled biometric features of the valid user.

In the biometric authentication process as described above, a livenesstest is performed. In an example, the liveness test is performed eitherbefore or after the biometric authentication result is determined. Inanother example, the biometric authentication process and the livenesstest process are performed together. The liveness test is to testwhether an object being a test subject is an animate object, and todetermine whether an authentication means is genuine. For example, theliveness test tests whether a face shown in an image captured by thecamera 140 is a genuine face of a person or a fake face. The livenesstest is used to discriminate between inanimate objects, such as, forexample, a photo, paper, a video, a model, and a mask as fake means, andanimate objects, such as, for example, a real face of a person.

FIG. 2 illustrates examples of fake faces 210 and a genuine face 220.The electronic device 120 identifies the genuine face 220 in a testsubject image acquired by capturing a real user face, through a livenesstest. Further, the electronic device 120 identifies a fake face 210 in atest subject image acquired by capturing a user face displayed on a PCscreen or a smart phone screen, a user face in a photo, a user faceprinted on paper, a model of the user face, through a liveness test.

An invalid user may attempt to cause a false acceptance of a userauthentication system using spoofing techniques. For example, in a faceverification, the invalid user presents a color photo, a video, or amodel or mask of a face of a valid user to the camera 140, to cause afalse acceptance. The liveness test prevents a false acceptance byfiltering authentication attempts (or spoofing attacks) using asubstitute such as a photo, a video, a mask, or a model. In response tothe determination that the authentication subject is an inanimate objectas a result of the liveness test, the electronic device 120 does notmove to a user authentication operation of comparing the input objectwith an enrolled object to determine matching therebetween, ordetermines that the user authentication is finally failed irrespectiveof a user authentication result.

Referring back to FIG. 1 , the electronic device 120 performs one of theliveness test and the biometric authentication, or performs both theliveness test and the biometric authentication. The electronic device120 is, for example, various devices and/or systems such as, forexample, a smart phone, a mobile phone, a wearable device, (such as, aring, a watch, a pair of glasses, glasses-type device, a bracelet, anankle bracket, a belt, a necklace, an earring, a headband, a helmet, adevice embedded in the cloths, or an eye glass display (EGD)), acomputing device, for example, a server, a laptop, a notebook, asubnotebook, a netbook, an ultra-mobile PC (UMPC), a tablet personalcomputer (tablet), a phablet, a mobile internet device (MID), a personaldigital assistant (PDA), an enterprise digital assistant (EDA), an ultramobile personal computer (UMPC), a portable lab-top PC, electronicproduct, for example, a robot, a digital camera, a digital video camera,a portable game console, an MP3 player, a portable/personal multimediaplayer (PMP), a handheld e-book, a global positioning system (GPS)navigation, a personal navigation device, portable navigation device(PND), a handheld game console, an e-book, a television (TV), a smartTV, a smart appliance, a home appliance, a smart home device, abiometric door lock, a security device, a security device for gatecontrol, a smart speaker, a robot, various Internet of Things (IoT)devices, a kiosk a vehicle starting device, or a vehicle opening device,and may be performed by an application, middleware, or an operatingsystem installed on the device, or a program of a server interoperatingwith the corresponding application on the device.

For the liveness test and/or biometric authentication process, theelectronic device 120 uses the radar sensor 130 and the image sensor140. In general, the radar sensor 130 does not have a great powerconsumption, whereas the image sensor 140 has a relatively greater powerconsumption. The radar sensor 130 is always or periodically activatedfor always-on sensing. The radar sensor 130 operates in a communicationmodule that provides a radar function. In an example, the electronicdevice 120 automatically senses an approach of the user 110 using theradar sensor 130. When the electronic device 120 senses an approach ofthe user 110, a liveness test based on radar data sensed by the radarsensor 130 is performed, and the image sensor 140 is activated, if aresult of the liveness test satisfies a condition. In an example, theelectronic device 120 secondarily performs a liveness test based on aface image of the user 110 acquired through the image sensor 140.

If only an image sensor is used without a radar sensor, it is difficultto keep the image sensor, which consumes a greater amount of power,always activated. Thus, the image sensor may be activated through apredetermined trigger motion such as, for example, pressing aswitch/button, or touching or moving a screen, to perform theauthentication process. In this example, it is impossible to perform anauthentication through always-on sensing. Further, due to thecharacteristics of the image sensor, the performance of the image sensorvaries depending on the surrounding lighting environment. Thus, aliveness test using only the image sensor is not robust againsttwo-dimensional (2D) spoofing attacks using a photo or a screen ormodel/mask-based 3D spoofing attacks.

However, a liveness test apparatus and a liveness test method describedherein may perform a liveness test using the radar sensor 130 and theimage sensor 140, thereby overcoming the drawbacks described above. Theelectronic device 120 performs always-on sensing using the radar sensor130 having a relatively small power consumption and performs a livenesstest using the radar data obtained from the radar sensor 130, therebyovercoming the weakness of the image sensor 140 being vulnerable to thesurrounding lighting environment. Further, by performing the livenesstest based on the radar data including the 3D shape information andmaterial property information of an object, the electronic device 120may robustly handle 2D spoofing attacks and 3D spoofing attacks. Indoing so, spoofing-based false acceptance may be effectively blocked,and the accuracy of a liveness test and a biometric authentication mayimprove.

FIG. 3 illustrates an example of an electronic device with a radarsensor and a camera sensor.

Referring to FIG. 3 , the electronic device 120 includes an image sensor140 and radar sensors 310. There may be one or more image sensors 140and one or more radar sensors 310. The image sensor 140 is a sensorconfigured to acquire image data and includes, for example, a colorsensor or an infrared (IR) sensor. The radar sensors 310 are sensorsconfigured to obtain radar data based on reception signals and disposedat various positions in the electronic device 120. A communicationmodule included in the electronic device 120 may perform the function ofthe radar sensors 310. For example, a communication module providing theIEEE 802.11 ad/ay communication technology may provide the function ofthe radar sensors 310.

The radar sensors 310 transmit transmission signals through transmissionantennas and obtain reflection signals being the transmitted signalsthat are reflected by an object, through reception antennas. The radarsensors 310 include one or more transmission antennas and one or morereception antennas. In an example, the radar sensors 310 may include aplurality of transmission antennas and a plurality of reception antennasand perform a multiple-input and multiple-output (MIMO) function usingthe transmission antennas and the reception antennas.

Whether there is an object, and a distance to the object may bedetermined based on temporal differences between the transmissionsignals of the radar sensors 310 and the reception signals that arereceived after being reflected by the object. Further, by analyzing thereception signals obtained through the plurality of reception antennasof the radar sensors 310, 3D shape information and material propertyinformation of the object may be extracted. For example, if the objectis a face, features such as the size of the face, a 3D shape, areflection property, a depth of a main point of the face, and a distancebetween main points of the face may be estimated by analyzing thereception signals.

FIG. 4 illustrates an example of a liveness test process. The operationsin FIG. 4 may be performed in the sequence and manner as shown, althoughthe order of some operations may be changed or some of the operationsomitted without departing from the spirit and scope of the illustrativeexamples described. Many of the operations shown in FIG. 4 may beperformed in parallel or concurrently. The blocks of the liveness testprocess of FIG. 4 , and combinations of the blocks, are performed by aliveness test apparatus. In an example, the liveness test apparatus isimplemented by special purpose hardware-based computer, and devices suchas a processor, that perform the specified functions, or combinations ofspecial purpose hardware and computer instructions included in theliveness test apparatus. In addition to the description of FIG. 4 below,the descriptions of FIGS. 1-3 is also applicable to FIG. 4 and areincorporated herein by reference. Thus, the above description may not berepeated here.

Referring to FIG. 4 , in operation 410, a liveness test apparatus, whichis an apparatus for performing a liveness test, determines whether thereis a test subject using a radar sensor. The liveness test apparatusimplements always-on sensing technology using the radar sensor that mayoperate with low power and at short intervals. The liveness testapparatus may perform the function of the radar sensor through acommunication module. In an example, the radar sensor uses only aportion of all antennas to determine whether there is a test subject.For example, the radar sensor monitors whether there is a test subjectusing a transmission antenna and a reception antenna as a pair.

The liveness test apparatus examines whether there is a test subject byanalyzing radar data obtained through the radar sensor. The livenesstest apparatus calculates a time of flight between a transmission signaland a reception signal based on the radar data and estimates a distanceto the test subject based on the calculated time of flight. If thedistance to the test subject is less than or equal to a threshold value,the liveness test apparatus determines that there is a test subject. Theliveness test apparatus determines whether there is a test subject byanalyzing by-distance signal intensities of the reception signal.

When it is determined that there is no test subject, the liveness testapparatus continually or continuously examines whether there is a testsubject using the radar sensor. When it is determined that a testsubject is present, the liveness test apparatus performs a firstliveness test using the radar sensor, in operation 420. The livenesstest apparatus obtains radar data using the radar sensor and performsthe first liveness test using the obtained radar data. In an example,the radar sensor obtains more detailed radar data using a greater numberof antennas that those used in operation 410. The liveness testapparatus extracts features related to the shape of the test subject,curvatures of the test subject, the size of the test subject, materialproperties of the test subject, a direction in which the test subject ispositioned, and a distance to the test subject, by analyzing the radardata. The radar sensor transmits electromagnetic waves, and receptionsignals obtained when the electromagnetic waves are reflected by thetest subject include information associated with material properties.For example, the information associated with material properties changesbased on whether the material of the test subject is metal, plastic, orreal human skin. Thus, the liveness test apparatus effectivelydetermines whether the test subject is a real person based on theinformation associated with material properties included in the radardata.

When it is determined that the test subject being is not an animateobject but a fake object as a result of the first liveness test, theliveness test apparatus returns to operation 410 and continually orcontinuously monitors whether there is a test subject. When it isdetermined that the test subject is an animate object as a result of thefirst liveness test, the liveness test apparatus activates an imagesensor, in operation 430. For example, a wake-up function for the imagesensor is performed. The activated image sensor acquires image data. Forexample, the image sensor acquires one or more photos or videos.

In operation 440, the liveness test apparatus detects a region ofinterest (for example, a face region) in the image data. In someexamples, the radar data may be used to detect the region of interest.For example, in response to a plurality of regions of interest beingdetected from the image data, a direction or an area in which the testsubject is detected may be identified based on the radar data, and aregion of interest positioned in the identified direction or area may bedetermined to be a final region of interest. In another example, aregion of interest detected in the image data may be corrected based oninformation (for example, a direction, a distance, or the size) relatedto the test subject in the radar data.

In operation 450, the liveness test apparatus performs a second livenesstest based on the region of interest. The liveness test apparatusperforms the second liveness test using a liveness test model. Forexample, pixel value information related to the region of interest isinput into the liveness test model, and the liveness test model providesa score (for example, an expected value or a probability value)indicating the likelihood of the test subject shown in the region ofinterest corresponding to an animate object. If the score is greaterthan a threshold value, the test subject is determined to be an animateobject. If the score is less than or equal to the threshold value, thetest subject is determined to be a fake object.

The liveness test model described herein may be, for example, a neuralnetwork model configured to output a value calculated by internalparameters based on input data. For example, the liveness test modelprovides a score indicating a feature value, a probability value, or avalue that a face object being a test subject corresponds to a genuineface or a fake face, based on the input data. The score is a value beinga standard for determining the liveness of the test subject. Forexample, the liveness test model may be based on a deep convolutionalneural network (DCNN) model. In an example, the DCNN model includes aconvolution layer, a pooling layer, and a fully connected layer, andprovides information for determining the liveness from the input datainput into the liveness test model through a computing process performedby each layer. The DCNN model is merely provided as an example. Theliveness test model may be based on a neural network model of astructure other than that of the DCNN model.

In another example, the liveness test apparatus calculates a similarityby comparing the result of the first liveness test and a result ofdetecting the region of interest, and determines the test subject to bean animate object if the calculated similarity is higher than areference value or determines the test subject to be a fake object ifthe similarity is less than or equal to the reference value. Theliveness test apparatus compares the direction of the test subject, thedistance to the test subject, and the size of the test subject, and setsa reference value in view of an error rate for each sensor and aresolution for each sensor.

When the test subject is determined to not be an animate object but afake object as a result of the second liveness test, the liveness testapparatus returns to operation 410 and continually or continuouslymonitors whether there is a test subject. When the test subject isdetermined to be an animate object as a result of the second livenesstest, the liveness test apparatus performs a third liveness test usingboth the radar data and the image data, in operation 460. For the thirdliveness test, the liveness test apparatus obtains detailed radar datausing a plurality of antennas. Preferably, the radar data may beobtained using a maximum number of antennas or using a relatively widefrequency band. The liveness test apparatus obtains the radar data usingdifferent channels or a plurality of polarization antennas. Whendifferent channels are used, frequency-based features are extracted fromthe radar data. Polarization characteristic-based features are extractedthrough the polarization antennas.

The liveness test apparatus extracts features from the radar data andfrom the image data and obtains a score indicating how likely the testsubject is to correspond to an animate object by inputting the extractedfeatures into the liveness test model. Features related to propagationreflection according to the medium of the test subject are extractedfrom the radar data, and features such as a distance between main parts(for example, both eyes), and the size/shape of a main part (forexample, an eye, a nose, or a mouth) are extracted from the image data.In another example, the liveness test apparatus generates combinationdata by combining the radar data and the image data and inputs thecombination data into the liveness test model. The liveness test modelprovides a score corresponding to the combination data. For example, theliveness test model is implemented as a single neural network model orimplemented as a plurality of neural network models.

In some examples, one of the first liveness test, the second livenesstest, and the third liveness test may be omitted.

The liveness test apparatus performs a low-power and high-performanceliveness test using always-on sensing technology through the processdescribed above. In particular, the liveness test apparatus operateswith low power and thus, effectively operates even on a mobile platform.Further, by using radar data and image data together, it is possible toreduce the variation in the performance of the liveness test caused bythe surrounding lighting environment and to effectively prevent 2Dspoofing attacks and 3D spoofing attacks.

FIG. 5 illustrates an example of controlling the activation of an imagesensor based on radar data.

Referring to FIG. 5 , in operation 510, the electronic device 120continually or continuously examines whether there is a test subjectusing the radar sensor 130. The radar sensor 130 periodically transmitsa signal. If the user 110 enters a predetermined area, the signaltransmitted from the radar sensor 130 is reflected by the user 110, andthe reflected signal is received by the radar sensor 130. The electronicdevice 120 determines whether there is a test subject close to and infront of the electronic device 120 by analyzing radar data includinginformation related to the received signal.

When it is determined that there is a test subject, the electronicdevice 120 performs a first liveness test with respect to the user 110based on the radar data obtained by the radar sensor 130. For example,if the face of the user 110 approaches the electronic device 120, theelectronic device 120 automatically recognizes the presence of the faceand first performs a liveness test for a face verification process. Theelectronic device 120 extracts features (for example, reflectionfeatures, or 3D shape features) of the test subject from the radar dataand determines whether the extracted features correspond to features ofan animate object.

In response to a result of the first liveness test satisfying a firstcondition, for example, in response to the determination that thefeatures extracted from the radar data correspond to the features of ananimate object, the electronic device 120 activates the image sensor140, in operation 520. The activated image sensor 140 acquires imagedata related to the face of the user 110, and the electronic device 120performs a second liveness test based on the acquired image data. Inresponse to the result of the first liveness test not satisfying thefirst condition, the electronic device 120 maintains a current state(for example, a state of being locked). Accordingly, if an actual faceof the user 110 is within a field of view (FOV) of the image sensor 140,a face recognition function operates. However, if a medium for aspoofing attack is within the FOV of the image sensor 140, the facerecognition function does not operate.

In another example, When it is determined that there is a test subjectthrough the radar sensor 130, the image sensor 140 is activated, and afirst liveness test is performed based on image data acquired by theimage sensor 140, rather than performing the first liveness test basedon the radar data.

The examples described with reference to FIG. 5 may also be performed bythe liveness test apparatus described herein, rather than the electronicdevice 120.

FIG. 6 illustrates an example of detecting a face region in image data.

Referring to FIG. 6 , a liveness test apparatus detecting a face region620 in image data 610, when performing a liveness test based on theimage data 610. For example, the liveness test apparatus detects theface region 620 using, for example, a neural network, a Viola-Jonesdetector, or a Haar-based cascade AdaBoost classifier. The liveness testapparatus detects feature points 630 corresponding to endpoints of botheyes, a nose tip point, and both corner points of a mouth in the faceregion 620. For example, the liveness test apparatus detects the featurepoints 630 using techniques such as Speeded Up Robust Features (SURF),Active Appearance Model (AAM), Active Shape Model (ASM), SupervisedDescent Method (SDM), or deep learning. The liveness test apparatusperforms image processing such as image scaling or image warping on theface region 620 based on the feature points 630 and performs theliveness test based on the processed face region.

FIGS. 7A and 7B illustrate examples of performing a third liveness test.

Referring to FIG. 7A, a third liveness test is performed based on bothradar data 710 and image data 720. The radar data 710 is input into aliveness test model 730, and the liveness test model 730 outputs a firstscore corresponding to the radar data 710. The image data 720 is inputinto a liveness test model 740, and the liveness test model 740 outputsa second score corresponding to the image data 720. In this example, theliveness test model 730 for the radar data 710 and the liveness testmodel 740 for the image data 720 are separately provided. A livenesstest apparatus determines a result 750 of the third liveness test withrespect to a test subject based on the first score and the second score.

FIG. 7B illustrates an example of using a single liveness test model760. When the liveness test apparatus performs a third liveness test,the radar data 710 and the image data 720 are input into the singleliveness test model 760, and the liveness test model 760 outputs a scorewith respect to the test subject. The liveness test apparatus determinesa result 770 of the third liveness test with respect to a test subjectbased on the score obtained through the liveness test model 760. As inthis example, the liveness test models 730 and 740 of FIG. 7A may bereplaced with the single integrated neural network model 760.

FIG. 8 illustrates an example of operations of a liveness test method.The operations in FIG. 8 may be performed in the sequence and manner asshown, although the order of some operations may be changed or some ofthe operations omitted without departing from the spirit and scope ofthe illustrative examples described. Many of the operations shown inFIG. 8 may be performed in parallel or concurrently. The blocks ofliveness test method is performed by the liveness test apparatusdescribed herein. In an example, the liveness test apparatus isimplemented by special purpose hardware-based computer, and devices suchas a processor, that perform the specified functions, or combinations ofspecial purpose hardware and computer instructions included in theliveness test apparatus. In addition to the description of FIG. 8 below,the descriptions of FIGS. 1-7 is also applicable to FIG. 8 and areincorporated herein by reference. Thus, the above description may not berepeated here.

Referring to FIG. 8 , in operation 810, the liveness test apparatusexamines whether there is a test subject using a radar sensor. Theliveness test apparatus continually or continuously obtains radar datafrom the radar sensor and examines whether there is a test subject basedon the obtained radar data. For example, the liveness test apparatusmonitors whether there is a human face based on the radar data. Theliveness test apparatus obtains the radar data using a portion ofantennas included in the radar sensor and extracts intensity features ofby-distance reception signals from the obtained radar data. The livenesstest apparatus estimates information related to the size and shape of atest subject based on the extracted intensity features and examineswhether there is a test subject based on the estimated information.

In operation 820, the liveness test apparatus determines whether thereis a test subject based on a result of the examining of operation 810.If there is no test subject, the liveness test apparatus returns tooperation 810 and continually or continuously examines whether a testsubject is present. When it is determined that a test subject ispresent, in operation 830, the liveness test apparatus performs a firstliveness test with respect to the test subject based on the radar dataobtained by the radar sensor.

In performing the first liveness test, the liveness test apparatusobtains radar data using a greater number of antennas of the radarsensor than those used in operation 810 and extracts features from theobtained radar data. For example, the liveness test apparatus extracts,from the radar data, one or more features such as a distance to the testsubject, the size of the test subject, a direction in which the testsubject is positioned, a material property of the test subject, and theshape of the test subject. The liveness test apparatus determines aresult of the first liveness test with respect to the test subject basedon the extracted features. Testing a liveness of the test subjectincludes determining whether the test subject is an animate genuineobject or an inanimate fake object.

In operation 840, the liveness test apparatus determines whether theresult of the first liveness test satisfies a first condition. Theresult of the first liveness test is determined to be a score indicatingthe likelihood of the test subject corresponding to an animate object,and whether the condition of score being greater than a threshold issatisfied.

When the result of the first liveness test does not satisfying the firstcondition, the liveness test apparatus returns to operation 810 andcontinually or continuously examines whether there is a test subject.When the result of the first liveness test satisfies the firstcondition, the liveness test apparatus acquires image data related tothe test subject using an image sensor, in operation 850. The livenesstest apparatus activates the image sensor and acquires the image datafrom the activated image sensor. As described above, the liveness testapparatus activates the image sensor if it is determined there is a testsubject using the radar sensor and the result of the first liveness testdetermined based on the radar data of the radar sensor satisfies acondition.

In operation 860, the liveness test apparatus performs a second livenesstest with respect to the test subject based on the acquired image data.The liveness test apparatus detects a face region of the test subject inthe image data and performs the second liveness test based on thedetected face region.

The liveness test apparatus detects the face region in the image datausing a Viola-Jones detector, a neural network trained to detect a faceregion, or a Haar-based cascade AdaBoost classifier. However, examplesare not limited thereto. The liveness test apparatus may detect the faceregion in the image data using various face region detection techniques.For example, the liveness test apparatus detects facial landmarks in theimage data and detects a bounding region including the detectedlandmarks, as the face region.

In an example, the radar data obtained by the radar sensor may be usedto detect the face region. For example, when a plurality of face regionsare detected in the image data, the face region that is subject for thesecond liveness test is determined based on the position of the testsubject or a direction faced by the test subject, which is determinedfrom the radar data.

The liveness test apparatus determines a score with respect to the testsubject using a liveness test model that receives an image of the faceregion as an input, and determines the determined score to be a resultof the second liveness test.

In operation 870, the liveness test apparatus determines whether theresult of the second liveness test satisfies a second condition. Theresult of the second liveness test is determined to be a scoreindicating how likely the test subject is to correspond to an animateobject, and whether the condition that the score is greater than athreshold value is satisfied.

When the second liveness test does not satisfy the second condition, theliveness test apparatus returns to operation 810 and continually orcontinuously examines whether there is a test subject. When the secondliveness test satisfies the second condition, in operation 880, theliveness test apparatus performs a third liveness test with respect tothe test subject based on the radar data obtained by the radar sensorand the image data acquired by the image sensor.

The liveness test apparatus extracts a first feature based on pixelsvalues of pixels included in the face region of the image data for thesecond liveness test and extracts a second feature from the radar dataobtained using the radar sensor. The first feature and the secondfeature are extracted using a liveness test model. The liveness testapparatus determines the result of the third liveness test based on theextracted first feature and the extracted second feature.

The liveness test apparatus extracts the first feature based on thepixel values of the pixels included in the face region of the imagedata. The liveness test apparatus obtains the radar data using the radarsensor and extracts the second feature from the obtained radar data. Forexample, the liveness test apparatus obtains radar data using the radarsensor including a plurality of polarization antennas or obtains radardata for each of a plurality of channels using the radar sensor. Theliveness test apparatus extracts a channel-based signal feature as thesecond feature from the obtained radar data. The first feature and thesecond feature are extracted using a liveness test model. The livenesstest apparatus determines a result of the third liveness test based onthe extracted first feature and the extracted second feature.

In another example, the liveness test apparatus generates combinationdata by combining the radar data and the image data, extracts a featurefrom the combination data, and determines the result of the thirdliveness test based on the extracted feature. When the third livenesstest satisfies a third condition, the liveness test apparatus determinesthe test subject to be an animate object.

The liveness test apparatus performs a control operation in response tothe result of the third liveness test with respect to the test subject.In an example, when the test subject is determined to be an animateobject, the liveness test apparatus generates a control signal torequest execution of a user authentication procedure. In anotherexample, when the test subject is determined to not be an animate objectbut a fake object, the liveness test apparatus generates a controlsignal to block an access of the user, without requesting execution ofthe user authentication process. In another example, the liveness testapparatus returns to operation 810 and continues the examinationregarding whether there is a test subject.

In some examples, one of the first liveness test, the second livenesstest, and the third liveness test may be omitted from the presentliveness test method.

FIG. 9 illustrates an example of operations of a liveness test method.The operations in FIG. 9 may be performed in the sequence and manner asshown, although the order of some operations may be changed or some ofthe operations omitted without departing from the spirit and scope ofthe illustrative examples described. Many of the operations shown inFIG. 9 may be performed in parallel or concurrently. The blocks ofliveness test method is performed by the liveness test apparatusdescribed herein. In an example, the liveness test apparatus isimplemented by special purpose hardware-based computer, and devices suchas a processor, that perform the specified functions, or combinations ofspecial purpose hardware and computer instructions included in theliveness test apparatus. In addition to the description of FIG. 8 below,the descriptions of FIGS. 1-7 is also applicable to FIG. 8 and areincorporated herein by reference. Thus, the above description may not berepeated here.

Referring to FIG. 9 , in operation 910, the liveness test apparatusexamines whether there is a test subject using a radar sensor. Theliveness test apparatus continually or continuously obtains radar datafrom the radar sensor like the always-on sensing function, and monitorsthe obtained radar data until a test subject is detected based on theradar data.

In operation 920, the liveness test apparatus determines whether thereis a test subject based on a result of the examining of operation 910.When it is determined that there is no test subject, the liveness testapparatus returns to operation 910 and continually or continuouslyexamines whether there is a test subject. When it is determined that atest subject is present, in operation 930, the liveness test apparatusacquires image data with respect to the test subject using an imagesensor. In response to the determination that there is a test subject,the liveness test apparatus activates the image sensor and acquires theimage data from the activated image sensor.

In operation 940, the liveness test apparatus performs a first livenesstest with respect to the test subject based on the image data. The firstliveness test in the present example corresponds to the second livenesstest described in operation 860 of FIG. 8 .

In operation 950, the liveness test apparatus determines whether aresult of the first liveness test satisfies a first condition. When theresult of the first liveness test does not satisfy the first condition,the liveness test apparatus returns to operation 910.

When the result of the first liveness test satisfies the firstcondition, the liveness test apparatus performs a second liveness testwith respect to the test subject based on the radar data obtained by theradar sensor and the image data acquired by the image sensor, inoperation 960. The second liveness test in the present examplecorresponds to the third liveness test described in operation 880 ofFIG. 8 .

In an example, the liveness test apparatus extracts a first featurebased on pixel values of pixels included in a face region of the imagedata for the second liveness test and extracts a second feature from theradar data obtained using the radar sensor. The first feature and thesecond feature are extracted using a liveness test model. The livenesstest apparatus determines a result of the second liveness test based onthe extracted first feature and the extracted second feature.

In another example, the liveness test apparatus generates combinationdata by combining the radar data and the image data, extracts a featurefrom the combination data, and determines a result of the secondliveness test based on the extracted feature.

When the result of the second liveness test satisfies a second definedcondition, the liveness test apparatus determines the test subject to bean animate object. When the result of the second liveness test does notsatisfy the second condition, the liveness test apparatus returns tooperation 910 and continually or continuously monitors whether there isa test subject.

FIG. 10 illustrates an example of a process of training a liveness testmodel.

For the liveness test models described herein, parameters are determinedthrough a training process. Referring to FIG. 10 , during the trainingprocess, there are prepared numerous training data 1010 and label dataincluding desired value information respectively corresponding to thetraining data 110. In an example, the training data 1010 may be radardata, image data, or a combination thereof.

A training data selector 1020 selects training data to be used for acurrent training operation from among the training data 1010. Thetraining data selected by the training data selector 1020 is input intoa liveness test model 1030, and the liveness test model 1030 outputs aresult value corresponding to the training data through a computationprocess performed based on internal parameters. In an example, theliveness test model 1030 may be a neural network model and implementedas one or more neural network models.

A trainer 1040 updates the parameters of the liveness test model 1030based on the result value output from the liveness test model 1030. Inan example, the trainer 1040 calculates a loss incurred by a differencebetween the result value output from the liveness test model 1030 and adesired value included in the label data, and trains the liveness testmodel by adjusting the parameters of the liveness test model 1030 toreduce the loss. Then, the trainer 1040 controls the training dataselector 1020 to select subsequent training data and trains the livenesstest model 1030 again based on the selected subsequent training data. Byiteratively performing the process as described above with respect toeach of the numerous training data 1010, the parameters of the livenesstest model 1030 are adjusted gradually as desired. In addition, thetrainer 1040 also trains the liveness test model 1030 using variousmachine learning algorithms.

FIG. 11 illustrates an example of a configuration of a liveness testapparatus.

Referring to FIG. 11 , a liveness test apparatus 1100 corresponds to theliveness test apparatus described herein. The liveness test apparatus1100 performs a liveness test based on radar data and/or image data. Theliveness test apparatus 1100 includes a processor 1110 and a memory1120. In some examples, the liveness test apparatus 1100 may furtherinclude at least one of a radar sensor 1130 and an image sensor 1140.

The radar sensor 1130 obtains the radar data through antennas. The radarsensor 1130 transmits signals through transmission antennas and receivesreflection signals being the transmitted signals that are reflected byan object, through reception antennas. In an example, the radar sensor1130 samples the signals received through the reception antennas andconverts the sampled signals into digital signals. Through the processdescribed above, the radar data is obtained. The image sensor 1140 is asensor configured to acquire image data and includes sensors such as,for example, a color sensor, an IR sensor, or a depth sensor.

The memory 1120 is connected to the processor 1110 and storesinstructions to be executed by the processor 1110, data to be computedby the processor 1110, or data processed by the processor 1110. Thememory 1120 includes computer-readable instructions. The processor 1420performs the above-described operations in response to the instructionsstored in the memory 1120 being executed in the processor 1110. Thememory 1120 is a volatile memory or a non-volatile memory. The memory1120 includes a large capacity storage medium such as a hard disk tostore the variety of data. Further details regarding the memory 1120 isprovided below.

The processor 1110 controls the overall function and operation of theliveness test apparatus 1100 and performs the one or more operationsrelated to a liveness test process described with reference to FIGS. 1to 10 . The processor 1110 performs a liveness test with respect to atest subject using at least one of the radar sensor 1130 and the imagesensor 1140.

In an example, the processor 1110 is configured to execute instructionsor programs, or to control the liveness test apparatus 1100. Theprocessor 1110 includes, for example, a central processing unit (CPU), aprocessor core, a multi-core processor, a reconfigurable processor, amulticore processor, a multiprocessor, an application-specificintegrated circuit (ASIC), and a field programmable gate array (FPGA),and/or a graphics processing unit (GPU), or any other type of multi- orsingle-processor configuration. In an example, the liveness testapparatus 1100 is connected to an external device via the one or more ofthe plurality of communication modules, and exchanges data. Furtherdetails regarding the processor 1110 is provided below.

In an example, the processor 1110 determines whether a test subject ispresent using the radar sensor 1130. The processor 1110 continually orcontinuously obtains the radar data from the radar sensor 1130 anddetermines whether there is a test subject based on the obtained radardata. When it is determined that a test subject is present, theprocessor 1110 performs a first liveness test with respect to the testsubject based on the radar data obtained by the radar sensor 1130. Whena result of the first liveness test satisfying a first condition, theprocessor 1110 activates the image sensor 1140 and acquires the imagedata from the activated image sensor 1140. Then, the processor 1110performs a second liveness test with respect to the test subject basedon the image data. When a result of the second liveness test satisfyinga second condition, the processor 1110 performs a third liveness testwith respect to the test subject based on the radar data obtained by theradar sensor 1130 and the image data acquired by the image sensor 1140.When a result of the third liveness test satisfies a third condition,the processor 1110 finally determines the test subject to be an animateobject.

In another example, the processor 1110 examines whether there is a testsubject using the radar sensor 1130. When it is determined that a testsubject is present, the processor 110 activates the image sensor 1140and acquires image data from the image sensor 1140. The processor 1110performs a first liveness test with respect to the test subject based onthe acquired image data. When a result of the first liveness testsatisfies a first condition, the processor 1110 performs a secondliveness test with respect to the test subject based on radar dataobtained by the radar sensor 1130 and the image data acquired by theimage sensor 1140. When a result of the second liveness test satisfies asecond condition, the processor 1110 determines the test subject to bean animate object.

The processor 1110 generates a control signal based on a final result ofthe liveness test. For example, when the test subject is determined tobe an inanimate object (or fake object) as a result of the livenesstest, the processor 1110 generates a control signal to block an accessof the object or to reject execution of a requested function.

FIG. 12 illustrates an example of a configuration of an electronicdevice.

Referring to FIG. 12 , an electronic device 1200 may correspond to theelectronic device described herein and perform the function of theliveness test apparatus 1100 of FIG. 11 . Thus, the above description ofFIG. 12 may not be repeated here. The electronic device 1200 includes aprocessor 1210, a memory 1220, a radar sensor 1230, an image sensor1240, a storage device 1250, an input device 1260, an output device1270, and a communication device 1180. The elements of the electronicdevice 1200 communicate with each other through a communication bus1290.

The processor 1210 executes instructions and functions to perform aliveness test and/or a biometric authentication. For example, theprocessor 1210 processes the instructions stored in the memory 1220 orthe storage device 1250. The processor 1210 performs the one or moreoperations described with reference to FIGS. 1 to 11 .

The memory 1220 stores the instructions to be executed by the processor1210 and information to be used to perform a liveness test and/or abiometric authentication. The memory 1220 may include acomputer-readable storage medium.

The radar sensor 1230 obtains radar data through transmission of signalsand reception of signals. The image sensor 1240 acquires image data. Inan example, the image sensor 1240 includes a color sensor, an IR sensor,and a depth sensor.

The storage device 1250 may include a computer-readable storage medium.The storage device 1250 may store a greater quantity of information thanthe memory 1220 and store the information for a relatively long time.For example, the storage device 1250 may include a magnetic hard disk,an optical disk, a flash memory, or a floppy disk. Further detailsregarding the storage device 1250 is provided below

The input device 1260 receives an input from a user through a haptic,video, audio, or touch input. For example, the input device 1260 mayinclude a keyboard, a mouse, a touch screen, a microphone, or any deviceconfigured to detect an input from a user and transmit the detectedinput to the electronic device 1200.

The output device 1270 provides an output of the electronic device 1200to the user through a visual, audio, or haptic channel. The outputdevice 1270 may include, for example, a display, a touch screen, aspeaker, a vibration generator, or any device configured to provide anoutput to the user. The communication device 1280 communicates with anexternal device through a wired or wireless network.

The liveness test apparatus, the liveness test apparatus 1100, 1200 andother apparatuses, units, modules, devices, and other componentsdescribed herein with respect to FIGS. 1-12 are implemented by hardwarecomponents. Examples of hardware components that may be used to performthe operations described in this application where appropriate includecontrollers, sensors, generators, drivers, memories, comparators,arithmetic logic units, adders, subtractors, multipliers, dividers,integrators, and any other electronic components configured to performthe operations described in this application. In other examples, one ormore of the hardware components that perform the operations described inthis application are implemented by computing hardware, for example, byone or more processors or computers. A processor or computer may beimplemented by one or more processing elements, such as an array oflogic gates, a controller and an arithmetic logic unit, a digital signalprocessor, a microcomputer, a programmable logic controller, afield-programmable gate array, a programmable logic array, amicroprocessor, or any other device or combination of devices that isconfigured to respond to and execute instructions in a defined manner toachieve a desired result. In one example, a processor or computerincludes, or is connected to, one or more memories storing instructionsor software that are executed by the processor or computer. Hardwarecomponents implemented by a processor or computer may executeinstructions or software, such as an operating system (OS) and one ormore software applications that run on the OS, to perform the operationsdescribed in this application. The hardware components may also access,manipulate, process, create, and store data in response to execution ofthe instructions or software. For simplicity, the singular term“processor” or “computer” may be used in the description of the examplesdescribed in this application, but in other examples multiple processorsor computers may be used, or a processor or computer may includemultiple processing elements, or multiple types of processing elements,or both. For example, a single hardware component or two or morehardware components may be implemented by a single processor, or two ormore processors, or a processor and a controller. One or more hardwarecomponents may be implemented by one or more processors, or a processorand a controller, and one or more other hardware components may beimplemented by one or more other processors, or another processor andanother controller. One or more processors, or a processor and acontroller, may implement a single hardware component, or two or morehardware components. A hardware component may have any one or more ofdifferent processing configurations, examples of which include a singleprocessor, independent processors, parallel processors,single-instruction single-data (SISD) multiprocessing,single-instruction multiple-data (SIMD) multiprocessing,multiple-instruction single-data (MISD) multiprocessing, andmultiple-instruction multiple-data (MIMD) multiprocessing.

The methods illustrated in FIGS. 1-12 that perform the operationsdescribed in this application are performed by computing hardware, forexample, by one or more processors or computers, implemented asdescribed above executing instructions or software to perform theoperations described in this application that are performed by themethods. For example, a single operation or two or more operations maybe performed by a single processor, or two or more processors, or aprocessor and a controller. One or more operations may be performed byone or more processors, or a processor and a controller, and one or moreother operations may be performed by one or more other processors, oranother processor and another controller. One or more processors, or aprocessor and a controller, may perform a single operation, or two ormore operations.

Instructions or software to control a processor or computer to implementthe hardware components and perform the methods as described above arewritten as computer programs, code segments, instructions or anycombination thereof, for individually or collectively instructing orconfiguring the processor or computer to operate as a machine orspecial-purpose computer to perform the operations performed by thehardware components and the methods as described above. In an example,the instructions or software includes at least one of an applet, adynamic link library (DLL), middleware, firmware, a device driver, anapplication program storing the liveness test method. In one example,the instructions or software include machine code that is directlyexecuted by the processor or computer, such as machine code produced bya compiler. In another example, the instructions or software includehigher-level code that is executed by the processor or computer using aninterpreter. Programmers of ordinary skill in the art can readily writethe instructions or software based on the block diagrams and the flowcharts illustrated in the drawings and the corresponding descriptions inthe specification, which disclose algorithms for performing theoperations performed by the hardware components and the methods asdescribed above.

The instructions or software to control computing hardware, for example,one or more processors or computers, to implement the hardwarecomponents and perform the methods as described above, and anyassociated data, data files, and data structures, may be recorded,stored, or fixed in or on one or more non-transitory computer-readablestorage media. Examples of a non-transitory computer-readable storagemedium include read-only memory (ROM), random-access programmable readonly memory (PROM), electrically erasable programmable read-only memory(EEPROM), random-access memory (RAM), dynamic random access memory(DRAM), static random access memory (SRAM), flash memory, non-volatilememory, CD-ROMs, CD-Rs, CD+Rs, CD-RWs, CD+RWs, DVD-ROMs, DVD-Rs, DVD+Rs,DVD-RWs, DVD+RWs, DVD-RAMs, BD-ROMs, BD-Rs, BD-R LTHs, BD-REs, blue-rayor optical disk storage, hard disk drive (HDD), solid state drive (SSD),flash memory, card type memory such as multimedia card, secure digital(SD) card, or extreme digital (XD) card, magnetic tapes, floppy disks,magneto-optical data storage devices, optical data storage devices, harddisks, solid-state disks, and any other device that is configured tostore the instructions or software and any associated data, data files,and data structures in a non-transitory manner and providing theinstructions or software and any associated data, data files, and datastructures to a processor or computer so that the processor or computercan execute the instructions. In one example, the instructions orsoftware and any associated data, data files, and data structures aredistributed over network-coupled computer systems so that theinstructions and software and any associated data, data files, and datastructures are stored, accessed, and executed in a distributed fashionby the one or more processors or computers.

While this disclosure includes specific examples, it will be apparentafter an understanding of the disclosure of this application thatvarious changes in form and details may be made in these exampleswithout departing from the spirit and scope of the claims and theirequivalents. The examples described herein are to be considered in adescriptive sense only, and not for purposes of limitation. Descriptionsof features or aspects in each example are to be considered as beingapplicable to similar features or aspects in other examples. Suitableresults may be achieved if the described techniques are performed in adifferent order, and/or if components in a described system,architecture, device, or circuit are combined in a different manner,and/or replaced or supplemented by other components or theirequivalents. Therefore, the scope of the disclosure is defined not bythe detailed description, but by the claims and their equivalents, andall variations within the scope of the claims and their equivalents areto be construed as being included in the disclosure.

What is claimed is:
 1. A liveness test method, comprising: determining apresence of a subject using a radar sensor; performing a first livenesstest on the subject based on radar data obtained by the radar sensor, inresponse to the subject being present; acquiring image data of thesubject using an image sensor, in response to a result of the firstliveness test satisfying a first condition; detecting, using the radardata, a face region of the subject in the image data; and performing,based on the detected face region of the subject in the image data, asecond liveness test on the subject.
 2. The liveness test method ofclaim 1, wherein the determining comprises determining whether there isa human face based on the radar data.
 3. The liveness test method ofclaim 2, wherein the radar data is obtained using a portion of antennasin the radar sensor.
 4. The liveness test method of claim 1, wherein thedetermining comprises: extracting an intensity feature of each distancefrom reception signal of the radar data; and determining the presence ofthe subject based on the intensity feature.
 5. The liveness test methodof claim 1, wherein the determining comprises: continuously obtainingdata from the radar sensor; and determining whether there is a subjectbased on the continuously obtained data.
 6. The liveness test method ofclaim 1, wherein the performing of the first liveness test comprises:extracting a feature from the radar data; and determining a result ofthe first liveness test on the subject based on the extracted feature.7. The liveness test method of claim 6, wherein the extracting comprisesextracting, from the radar data, any one or any combination of adistance to the subject, a size of the subject, a direction in which thesubject is positioned, and a shape of the subject.
 8. The liveness testmethod of claim 1, wherein the acquiring comprises: activating the imagesensor, in response to the result of the first liveness test satisfyingthe first condition; and acquiring the image data from the activatedimage sensor.
 9. The liveness test method of claim 1, furthercomprising: performing a third liveness test on the subject based on theradar data and the image data, in response to a result of the secondliveness test satisfying a second condition.
 10. The liveness testmethod of claim 9, wherein the performing of the third liveness testcomprises: extracting a first feature based on pixel values of pixelsincluded in a face region in the image data; obtaining another radardata using the radar sensor; extracting a second feature from theanother radar data; and determining a result of the third liveness testbased on the first feature and the second feature.
 11. The liveness testmethod of claim 10, wherein the another radar data is obtained using aplurality of polarization antennas of the radar sensor.
 12. The livenesstest method of claim 10, wherein the another radar data is obtained foreach of a plurality of channels using the radar sensor, and theextracting of the second feature comprises extracting a channel-basedsignal feature from the another radar data.
 13. A non-transitorycomputer-readable storage medium storing instructions that, whenexecuted by a processor, cause the processor to perform the livenesstest method of claim
 1. 14. The liveness test method of claim 1, whereinthe performing of the second liveness test comprises: detecting a regionof interest in the image data; extracting pixel value information fromthe region of interest; obtaining a score based on inputting the pixelvalue information into a liveness test model; and determining thesubject to be an animate object, in response to the score being greaterthan a threshold.
 15. A liveness test method, comprising: determining,using a radar data acquired by a radar sensor, a presence of a subject;acquiring, in response to the subject being present, image data of thesubject using an image sensor; detecting, using the radar data, aplurality of regions of interest within the image data; detecting, usingthe radar data, among the plurality of regions of interest within theimage data, a face region of the subject in the image data; performing,based on the detected face region of the subject in the image data, afirst liveness test on the subject based on the image data; andperforming a second liveness test on the subject based on the radar dataobtained by the radar sensor and the image data acquired by the imagesensor, in response to a result of the first liveness test satisfying afirst condition.
 16. The liveness test method of claim 15, wherein thedetermining comprises: continuously obtaining the radar data from theradar sensor; and determining whether the subject is present based onthe obtained radar data.
 17. The liveness test method of claim 15,wherein the acquiring comprises: activating the image sensor, inresponse to the determination that the subject is present; and acquiringthe image data from the activated image sensor.
 18. The liveness testmethod of claim 15, wherein the performing of the second liveness testcomprises: extracting a first feature based on pixel values of pixelsincluded in a face region in the image data; obtaining another radardata using the radar sensor; extracting a second feature from theanother radar data; and determining a result of the second liveness testbased on the first feature and the second feature.
 19. A liveness testapparatus, comprising: a radar sensor; an image sensor; and a processorconfigured to: determine a presence of a subject using the radar sensor,perform a first liveness test on the subject based on radar dataobtained by the radar sensor, in response to the subject being present,acquire image data of the subject using the image sensor, in response toa result of the first liveness test satisfying a first condition,detect, using the radar data, a face region of the subject in the imagedata, and perform, based on the detected face region of the subject inthe image data, a second liveness test on the subject.
 20. The livenesstest apparatus of claim 19, wherein the processor is further configuredto continuously obtain data from the radar sensor, and to determine thepresence of the subject based on the obtained data.
 21. The livenesstest apparatus of claim 19, wherein the processor is further configuredto activate the image sensor, in response to the result of the firstliveness test satisfying the first condition, and to acquire the imagedata from the activated image sensor.
 22. The liveness test apparatus ofclaim 19, wherein the radar sensor is configured to operate while beingincluded in a communication module.
 23. The liveness test apparatus ofclaim 19, wherein the processor is further configured to perform a thirdliveness test on the subject based on the radar data and the image data,in response to a result of the second liveness test satisfying a secondcondition.
 24. A liveness test apparatus, comprising: a radar sensor; animage sensor; and a processor configured to: determine a presence of asubject using the radar sensor, perform a first liveness test on thesubject based on radar data obtained by the radar sensor, in response tothe subject being present, acquire image data of the subject using theimage sensor, in response to a result of the first liveness testsatisfying a first condition, and perform a second liveness test on thesubject based on the image data, wherein the processor is furtherconfigured to perform a third liveness test on the subject based on theradar data and the image data, in response to a result of the secondliveness test satisfying a second condition.
 25. A liveness testapparatus, comprising: a radar sensor configured to obtain a radar data;an image sensor configured to obtain an image data; and a processorconfigured to: determine, using the radar data, whether a subject ispresent; acquire, in response to the subject being present, an imagedata of the subject; detect, using the radar data, a plurality ofregions of interest within the image data; detect, using the radar data,among the plurality of regions of interest, a face region of the subjectin the image data; perform, based on the detected face region of thesubject in the image data, a first liveness test on the subject; andperform a second liveness test on the subject based on the radar dataobtained by the radar sensor and the image data acquired by the imagesensor, in response to a result of the first liveness test satisfying afirst condition.
 26. A liveness test method, comprising: determining apresence of a subject using a radar sensor; performing a first livenesstest on the subject based on first radar data obtained by the radarsensor, in response to the subject being present; acquiring image dataof the subject using an image sensor, in response the first livenesstest satisfying a first threshold; performing a second liveness test onthe subject based on the image data; performing a third liveness test onthe subject based on second radar data and the image data, in responsethe second liveness test satisfying a second threshold.
 27. The livenesstest method of claim 26, wherein a number of antennas of the radarsensor used to obtain the second radar data is greater than a number ofantennas of the radar sensor used to obtain the first radar data. 28.The liveness test method of claim 26, wherein a number of antennas ofthe radar sensor used to obtain the first radar data is greater than anumber of antennas of the radar sensor used to determine the presence ofthe subject.